Joint Distributional Modeling with Cross-Correlation Based Features

In maximum-likelihood based speech recognition systems, it is important to accurately estimate the joint distribution of feature vectors given a particular acoustic model. In this work, we propose that by modeling the joint distribution of time-localized feature vectors and statistics relating those time-localized feature vectors to the relevant acoustic context, we can estimate information contained in the featurevector joint distribution without the accompanying theoretical or computational difficulties. We introduce the modcrossgram (MCG), a computational way of estimating short-time spectro-temporal correlation-based statistics that are informative about the feature-vector joint distribution. Using the standard hybrid ANN/HMM architecture, we compare a MCG-based speech recognition system with a more traditional one on an isolated word speech database. We show that, in the presence of noise, the MCG-based system achieves a significant reduction in word error rate over the standard system.